Investigating the Functioning of Rating Scales With Rasch Models.

The hypothesis implicit in the rating scale design is that the categories reflect increasing levels of the latent variable. Rasch models for ordered polytomous items include parameters, called thresholds, that allow for empirically testing this hypothesis. Failure of the thresholds to advance monotonically with the categories (a condition that is referred to as "threshold disordering") provides evidence that the rating scale is not functioning as intended. This work focuses on scales consisting of rather large numbers of categories, whose use is often recommended in the literature. Threshold disordering is observed in both an extended 8-point scale specially developed for the Patient Health Questionnaire-9 and the original 10-point scale of the Behavioral Religiosity Scale. The results of this work prompt practitioners not to take the functioning of the rating scale for granted, but to verify it empirically.

PsycAssist: A Web-Based Artificial Intelligence System Designed for Adaptive Neuropsychological Assessment and Training.

Assessing executive functions in individuals with disorders or clinical conditions can be challenging, as they may lack the abilities needed for conventional test formats. The use of more personalized test versions, such as adaptive assessments, might be helpful in evaluating individuals with specific needs. This paper introduces PsycAssist, a web-based artificial intelligence system designed for neuropsychological adaptive assessment and training. PsycAssist is a highly flexible and scalable system based on procedural knowledge space theory and may be used potentially with many types of tests. We present the architecture and adaptive assessment engine of PsycAssist and the two currently available tests: Adap-ToL, an adaptive version of the Tower of London-like test to assess planning skills, and MatriKS, a Raven-like test to evaluate fluid intelligence. Finally, we describe the results of an investigation of the usability of Adap-ToL and MatriKS: the evaluators perceived these tools as appropriate and well-suited for their intended purposes, and the test-takers perceived the assessment as a positive experience. To sum up, PsycAssist represents an innovative and promising tool to tailor evaluation and training to the specific characteristics of the individual, useful for clinical practice.

Development of a scale for capturing psychological aspects of physical-digital integration: relationships with psychosocial functioning and facial emotion recognition.

The present work aims at developing a scale for the assessment of a construct that we called "physical-digital integration", which refers to the tendency of some individuals not to perceive a clear differentiation between feelings and perceptions that pertain to the physical or digital environment. The construct is articulated in four facets: identity, social relationships, time-space perception, and sensory perception. Data from a sample of 369 participants were collected to evaluate factor structure (unidimensional model, bifactor model, correlated four-factor model), internal consistency (Cronbach's α, McDonald's ω), and correlations of the physical-digital integration scale with other measures. Results showed that the scale is valid and internally consistent, and that both the total score and the scores at its four subscales are worthy of consideration. The physical-digital integration scores were found to be differently associated with digital and non-digital behaviors, individuals' ability to read emotions in the facial expressions of others, and indicators of psychosocial functioning (anxiety, depression, and satisfaction with social relationships). The paper proposes a new measure whose scores are associated with several variables that may have relevant consequences at both individual and social levels.

Enhancing Computerized Adaptive Testing with Batteries of Unidimensional Tests.

The article presents a new computerized adaptive testing (CAT) procedure for use with batteries of unidimensional tests. At each step of testing, the estimate of a certain ability is updated on the basis of the response to the latest administered item and the current estimates of all other abilities measured by the battery. The information deriving from these abilities is incorporated into an empirical prior that is updated each time that new estimates of the abilities are computed. In two simulation studies, the performance of the proposed procedure is compared with that of a standard procedure for CAT with batteries of unidimensional tests. The proposed procedure yields more accurate ability estimates in fixed-length CATs, and a reduction of test length in variable-length CATs. These gains in accuracy and efficiency increase with the correlation between the abilities measured by the batteries.

Machine learning-decision tree classifiers in psychiatric assessment: An application to the diagnosis of major depressive disorder.

This work illustrates the advantages of using machine learning classifiers in psychiatric assessment. Machine learning-decision trees (ML-DTs) represent a new approach to scoring and interpreting psychodiagnostic test data that allows for increasing assessment accuracy and efficiency. The approach is outlined in an easy yet detailed way, and its application is illustrated on real psychodiagnostic test data. Specifically, cross-sectional data concerning nonclinical and clinical Japanese populations were taken from a panel registered with an internet survey company. Responses to the Patient Health Questionnaire-9 (PHQ-9) underwent receiver operating characteristic (ROC) curve, DSM algorithm, and ML-DT analyses. The results showed greater diagnostic accuracy for ML-DT (0.71-0.75) compared with the DSM algorithm (0.69) and ROC curves (0.70-0.71). Moreover, ML-DT enabled classifying participants as having or not having a diagnosis of depression using, on average, the information from 2.99 out of 9 items (SD = 1.35). The application showed that ML-DTs can provide information of high clinical value to integrate traditional psychometric methods. The resulting assessments are informative, accurate, and efficient.

Is the performance at the implicit association test sensitive to feedback presentation? A Rasch-based analysis.

The Implicit Association Test (IAT) is commonly used for the indirect assessment of psychological constructs. While the features of the IAT that might influence the performance of the respondents have been extensively investigated, the effect of informing the respondents about the correctness of their responses (i.e., feedback presentation) has been poorly addressed so far. The study addresses this issue by presenting an across-domain (implicit prejudice and food preference) Rasch-based analysis of IAT data obtained with and without feedback presentation. Results showed that speed was influenced by the interaction between feedback presentation and associative condition, whereas accuracy was influenced by the associative condition. This result varied across-domain. Results suggested that IATs administered with feedback presentation provide more accurate information on the construct of interest.

Italian Diagnostic Adaptive Behavior Scale: Reliability and diagnostic accuracy compared with the Vineland-II.

BACKGROUND: The Diagnostic Adaptive Behavior Scale (DABS) is a short scale with excellent properties to assess the conceptual, social, and practical adaptive behavior domains for the diagnosis of intellectual disability (ID) in individuals aged 4-21 years. AIMS: Investigate the test-retest and inter-respondent reliability of the Italian adaptation of the DABS, verify its diagnostic accuracy in identifying individuals with ID and excluding individuals with typical development (TD), and compare its psychometric properties to those of the Vineland-II. METHODS: Test-retest reliability: The same respondent completed the Italian DABS for the same assessed person at two separate times (n = 71). Inter-respondent reliability: Two respondents for the same assessed person completed the Italian DABS independently (n = 57). Diagnostic accuracy: The same respondent completed the Italian DABS and Vineland-II for the same assessed person (n = 378; 50 % ID, 50 % TD). RESULTS: Italian DABS test-retest and inter-respondent correlation coefficients were excellent. Italian DABS sensitivity was 86 % and specificity was 99 %, Italian DABS Areas Under the ROC Curves were excellent (or good, practical skill domain), and comparable to the results reported for the Vineland-II. CONCLUSIONS: The Italian DABS is an excellent measure to evaluate the adaptive behavior for ID diagnosis; it is comparable to the Vineland-II but being shorter, the Italian DABS requires less time to administer.

COVID-19 emergency: the influence of implicit attitudes, information sources, and individual characteristics on psychological distress, intentions to get vaccinated, and compliance with restrictive rules.

BACKGROUND: To limit the spread of the COVID-19 emergency, a massive vaccination program was implemented and restrictive measures were imposed on the population. However, the propensity to adhere to the vaccination program has struggled to take off. Moreover, complying with the restrictive rules and maintaining social distancing have been highly distressing for many individuals. PARTICIPANTS AND PROCEDURE: Italian participants (N = 140, females = 65%, mean age = 29.50, SD = 10.80) were presented with an online survey consisting of multiple-choice questions and two single-category implicit association tests (SC-IATs). One SC-IAT evaluated the tendency of participants to automatically associate personal protective equipment (PPE) and vaccines with safety or danger; the other evaluated their tendency to automatically associate social situations with good or bad. Multiple-choice questions explored individual, social, and environmental factors that were expected to contribute to vaccine propensity, compliance with restrictive rules, and feelings of distress. RESULTS: Using scientific information sources was related to implicitly associating PPE and vaccines with safety, which in turn was associated with the propensity to get the vaccine. Moreover, being female, young, unsatisfied with social relationships, having suffered health and economic consequences due to the pandemic, and having negative implicit attitudes toward social situations contributed to increasing feelings of distress. CONCLUSIONS: Communication may contribute to individuals' behavior and preferences and it can also be associated with implicit attitudes, becoming consequently one of the main leverages to reduce vaccine hesitancy. Recovery programs should prioritize the development of interventions aimed at fostering psychological well-being through the enhancement of social contacts.

An Item Response Theory-Based Scoring of the South Oaks Gambling Screen-Revised Adolescents.

The South Oaks Gambling Screen-Revised Adolescent (SOGS-RA) is one of the most widely used screening tools for problem gambling among adolescents. In this study, item response theory was used for computing measures of problem gambling severity that took into account how much information the endorsed items provided about the presence of problem gambling. A zero-inflated mixture two-parameter logistic model was estimated on the responses of 4,404 adolescents to the South Oaks Gambling Screen-Revised Adolescent to compute the difficulty and discrimination of each item, and the problem gambling severity level (θ score) of each respondent. Receiver operating characteristic curve analysis was used to identify the cutoff on the θ scores that best distinguished daily and nondaily gamblers. This cutoff outperformed the common cutoff defined on the sum scores in identifying daily gamblers but fell behind it in identifying nondaily gamblers. When screening adolescents to be subjected to further investigations, the cutoff on the θ scores must be preferred to that on the sum scores.

Perception of stress and cognitive efficiency in older adults with mild and moderate dementia during the COVID-19-related lockdown.

AIM: Many studies have been carried out with the aim of understanding the manifold effects of the novel coronavirus (Covid-19) on individuals' clinical and psychological states. This paper deals with perceived stress (PS) and cognitive efficiency (CE) in older adults with dementia during the first wave of the pandemic. The study also investigated the potential effects of PS and CE on participants' cognitive functioning. The modulating effect of cognitive reserve (CR) on these variables was also considered, given its well-known role in the onset and evolution of neurodegenerative diseases. METHOD: Thirty-eight older adults with mild/moderate dementia (mean age: 81.47 ± 5.05; mean MMSE pre-lockdown: 24.03 ± 3.14) were recruited for this study from March to May [4]. Two questionnaires, the Perceived Stress Scale (PSS) and the FLEI Mental Ability Questionnaire (FLEI), were administered to all participants by telephone every two weeks during lockdown (T1: early April, T2: mid-April, T3: early May). After lockdown, their neuropsychological and psychological profiles were assessed. Linear mixed-effects models were used to investigate changes over time. RESULTS: The level of PS increased at both Time 2 and Time 3 (f2 = 0.10). Cognitive functioning worsened during lockdown, resulting in lower scores at the post-lockdown evaluation (f2s = 0.09 and 0.06 for MMSE and ENB-2, respectively). The decrease in these scores was not associated with either PS or CE. Although the size of these effects was rather small, their clinical relevance is not negligible. CONCLUSION: Individuals with dementia seem to have experienced stress (S) during the first-wave of lockdown related to Covid-19. Cognition worsened during the pandemic, in accordance with the neurodegenerative nature of the disease, but it was unrelated to PS and CE.

Religious assessment in Italian older adults: psychometric properties of the Francis Scale of Attitude toward Christianity and the Behavioral Religiosity Scale.

OBJECTIVES: Studies have shown that religiosity (R) and spirituality (S) can positively impact older adults' life. Nevertheless, no validated tools for measuring these constructs in the older Italian population are currently available. This study investigates the psychometric properties of two of the most common measures of R and S in the literature: the Francis Scale of Attitude toward Christianity short form (FSACsf) and the Behavioral Religiosity Scale (BRS). METHOD: 313 older Italians completed the questionnaires anonymously. The functioning of the items and the response scales, the factor structure, age invariance and internal consistency of both scales were investigated. RESULTS: Each scale defines a substantively unidimensional variable. Moreover, items seem, on the whole, to be well formulated. The values of the Rasch-Andrich thresholds suggest that the response scale of the FSACsf was adequate for measuring spirituality. However, this was not the case for BRS, where the response categories needed to be collapsed to measure behavioral religiosity properly. Age invariance was confirmed for each scale. Cronbach's α, composite reliability and person separation reliability revealed both scales to be internally consistent. DISCUSSION: The results suggest that the scales have solid psychometric properties, and can therefore be considered valid, reliable tools for investigating religiosity and spirituality in older people.

Acetylcholinesterase inhibitors and cognitive stimulation, combined and alone, in treating individuals with mild Alzheimer's disease.

BACKGROUNDS: Acetylcholinesterase inhibitors (AChEI) and cognitive stimulation (CS) are the standard pharmacological and non-pharmacological treatments for Alzheimer's disease (AD). AIMS: The aim of this study was to investigate the effects of these treatments, alone or combined, on the neuropsychological profiles of patients with AD. METHODS: Forty participants were assigned to three groups receiving either only AChEI (n = 14), AChEI + CS (n = 15), or only CS (n = 11). Cognition was evaluated at baseline and after three months. Linear mixed-effects models were used to investigate differences among the treatments in terms of changes in the patients' neuropsychological profiles. RESULTS: Results, although preliminary because of the small sample size, suggest that a general improvement was found in patients who received AChEI + CS and those who received only CS compared with those who received only AChEI. Interestingly, individuals who received only CS showed a significant improvement in immediate memory recall than those who received only AChEI. Furthermore, the group receiving AChEI + CS showed an improvement in delayed recall than the other two groups. DISCUSSION: The combination of AChEI and CS seems to have the greatest benefit for patients with mild AD. More interestingly, CS alone is more effective than AChEI alone, even in improving memory, considered to be the "lost" cognitive domain in AD.

Extending the Basic Local Independence Model to Polytomous Data.

A probabilistic framework for the polytomous extension of knowledge space theory (KST) is proposed. It consists in a probabilistic model, called polytomous local independence model, that is developed as a generalization of the basic local independence model. The algorithms for computing "maximum likelihood" (ML) and "minimum discrepancy" (MD) estimates of the model parameters have been derived and tested in a simulation study. Results show that the algorithms differ in their capability of recovering the true parameter values. The ML algorithm correctly recovers the true values, regardless of the manipulated variables. This is not totally true for the MD algorithm. Finally, the model has been applied to a real polytomous data set collected in the area of psychological assessment. Results show that it can be successfully applied in practice, paving the way to a number of applications of KST outside the area of knowledge and learning assessment.

Explaining Symptoms in Systemic Therapy. Does Triadic Thinking Come Into Play?

The main aim of this study is to explore the breadth of the inference field and the type of etiopathogenetic contents of symptom explanations provided by the client and therapist in the first two psychotherapy sessions conducted using a systemic approach. Does the therapist use triadic explanations of psychopathology as suggested by her approach? And do clients resort almost exclusively to monadic and dyadic explanations as did the university students in our previous study? What kind of explanations do they propose? The coding system "1 to 3: from the monad to the triad" was applied to the transcripts of 25 individual systemic therapies conducted by the same therapist. This manual allows coding of the inference field of symptom explanations according to three categories: monadic, dyadic, and triadic. These three broad categories are also used to analyze the etiopathogenetic content of each explanation: traumatic, intrapersonal, and interpersonal. Our findings showed that clients and their therapist actually used different inference fields: clients resorted almost exclusively to monadic and dyadic explanations, whereas their therapist included the triadic explanatory level. Moreover, the therapist provided more interpersonal explanations than her clients. Hence, the dissonance between client and therapist about the inference fields - a crucial premise of one of the most accepted ideas of therapeutic change according to systemic therapies - is proven, at least among our participants. Thanks to this dissonance, clients and therapists can create a new story, potentially able to change clients' feelings, without disconfirming their emotions.

Does discrimination beat association in the IAT? The discrimination-association model reconceived.

The discrimination-association model (DAM; Stefanutti et al. 2013) disentangles two components underlying the responses to the implicit association test (IAT), which pertain to stimuli discrimination (the strength of the association of the stimuli with their own category) and automatic association (the strength of the association between targets and attributes). The assumption of the DAM that these two components sum into a single process generates critical drawbacks. The present work provides a new formulation of the model, called DAM-4C, in which stimuli discrimination and automatic association are separate, independent, and competing processes. Results of theoretical and simulation studies suggest that the DAM-4C outperforms the DAM. The IAT effect is found to vary with the association rates of the DAM-4C and not with those of the DAM. The parameters of the DAM-4C fitted on data from a Coca-Pepsi IAT are found to account for variance in brand attractiveness, taste preference, and cola choice that is not accounted for by the D score and the diffusion model. In addition, the association rates estimated on data from a Black-White IAT are in line with expectations.

A Comparison of Classical and Modern Measures of Internal Consistency.

Three measures of internal consistency - Kuder-Richardson Formula 20 (KR20), Cronbach's alpha (α), and person separation reliability (R) - are considered. KR20 and α are common measures in classical test theory, whereas R is developed in modern test theory and, more precisely, in Rasch measurement. These three measures specify the observed variance as the sum of true variance and error variance. However, they differ for the way in which these quantities are obtained. KR20 uses the error variance of an "average" respondent from the sample, which overestimates the error variance of respondents with high or low scores. Conversely, R uses the actual average error variance of the sample. KR20 and α use respondents' test scores in calculating the observed variance. This is potentially misleading because test scores are not linear representations of the underlying variable, whereas calculation of variance requires linearity. Contrariwise, if the data fit the Rasch model, the measures estimated for each respondent are on a linear scale, thus being numerically suitable for calculating the observed variance. Given these differences, R is expected to be a better index of internal consistency than KR20 and α. The present work compares the three measures on simulated data sets with dichotomous and polytomous items. It is shown that all the estimates of internal consistency decrease with the increasing of the skewness of the score distribution, with R decreasing to a larger extent. Thus, R is more conservative than KR20 and α, and prevents test users from believing a test has better measurement characteristics than it actually has. In addition, it is shown that Rasch-based infit and outfit person statistics can be used for handling data sets with random responses. Two options are described. The first one implies computing a more conservative estimate of internal consistency. The second one implies detecting individuals with random responses. When there are a few individuals with a consistent number of random responses, infit and outfit allow for correctly detecting almost all of them. Once these individuals are removed, a "cleaned" data set is obtained that can be used for computing a less biased estimate of internal consistency.

Development and psychometric properties of a short form of the Illness Denial Questionnaire.

BACKGROUND: Coping with chronic illness can be overwhelming for patients and caregivers, and may be inhibited by the denial mechanism, and therefore, denial represents a critical issue for health professionals. Assessing illness denial is far from easy, and brief tools suitable for medical settings are lacking. In this paper, the development of a short form of the Illness Denial Questionnaire (IDQ) for patients and caregivers is presented. METHODS: In study 1, the IDQ was administered to 118 patients and 83 caregivers to examine the internal structure of denial; then the properties of the items (DIF, fit, and difficulty) were evaluated according to the Rasch model in order to select the best items for the Illness Denial Questionnaire-Short Form (IDQ-SF). Study 2 included 202 participants (113 patients and 89 caregivers). The internal structure of the IDQ-SF was tested via confirmatory factor analysis (CFA). Reliability and concurrent validity were also studied using the Anxiety and Depression Questionnaire-Reduced Form (AD-R). RESULTS: The CFA showed a two-factor structure encompassing "Denial of negative emotions" and "Resistance to change". Results of the Rasch analyses led to the selection of 4 items for each dimension. The resulting IDQ-SF (8 items) showed a two-factor structure as well as good reliability and concurrent validity with AD-R. CONCLUSION: The IDQ-SF represents a valid tool for quickly evaluating the core of illness denial in patients and caregivers. This brief and easily administrable questionnaire allows health professionals to outline the presence and severity of illness denial in order to set individually tailored interventions.

New Perspectives in Computing the Point of Subjective Equality Using Rasch Models.

In psychophysics, the point of subject equality (PSE) is any of the points along a stimulus dimension at which a variable stimulus (visual, tactile, auditory, and so on) is judged by an observer to be equal to a standard stimulus. Rasch models have been found to offer a valid solution for computing the PSE when the method of constant stimuli is applied in the version of the method of transitions. The present work provides an overview of the procedures for computing the PSE using Rasch models and proposes some new developments. An adaptive procedure is described that allows for estimating the PSE of an observer without presenting him/her with all stimuli pairs. This procedure can be particularly useful in those situations in which psychophysical conditions of the individuals require that the number of trials is limited. Moreover, it allows for saving time that can be used to scrutinize the results of the experiment or to run other experiments. Also, the possibility of using Rasch-based fit statistics for identifying observers who gave unexpected judgments is explored. They could be individuals who, instead of carefully evaluating the presented stimuli pairs, gave random, inattentive, or careless responses, or gave the same response to many consecutive stimuli pairs. Otherwise, they could be atypical and clinically relevant individuals who deserve further investigation. The aforementioned developments are implemented using procedures and statistics that are well established in the framework of Rasch models. In particular, computerized adaptive testing procedures are used for efficiently estimating the PSE of the observers, whereas infit and outfit mean-squares statistics are used for detecting observers who gave unexpected judgments. Results of the analyses carried out on simulated data sets suggest that the proposed developments can be used in psychophysical experiments.

DscoreApp: A Shiny Web Application for the Computation of the Implicit Association Test D-Score.

Several options are available for computing the most common score for the Implicit Association Test, the so-called D-score. However, all these options come with some drawbacks, related to either the need for a license, for being tailored on a specific administration procedure, or for requiring a degree of familiarity with programming. By using the R shiny package, a user-friendly, interactive, and open source web application (DscoreApp) has been created for the D-score computation. This app provides different options for computing the D-score algorithms and for applying different cleaning criteria. Beyond making the D-score computation easier, DscoreApp offers the chance to have an immediate glimpse on the results and to see how they change according to different settings configurations. The resulting D-scores are immediately available and can be seen in easy-readable and interactive graphs, along with meaningful descriptive statistics. Graphical representations, data sets containing the D-scores, and other information on participants' performance are downloadable. In this work, the use of DscoreApp is illustrated on an empirical data set.

Validity and measurement invariance of the Unified Multidimensional Calling Scale (UMCS): A three-wave survey study.

The accumulation of scientific knowledge on calling is limited by the absence of a common theoretical and measurement framework. Many different models of calling have been proposed, and we do not know how much research results that refer to a specific model are generalizable to different theoretical accounts of calling. In this article, we investigate whether two leading models of calling tackle the same construct. The two models were merged into a comprehensive framework that measures calling across seven facets: Passion, Purposefulness, Sacrifice, Pervasiveness, Prosocial Orientation, Transcendent Summons, and Identity. We then developed the Unified Multidimensional Calling Scale (UMCS) drawing from previous published items. Across two surveys involving college students (N = 5886) and adult employees (N = 205) the UMCS was proved to be valid and reliable. We also observed that the UMCS is invariant across time and calling domains. Finally, we found that facets of calling have very different relationships with outcomes and concurrent measures, suggesting that results obtained with a smaller set of facets are not generalizable to the higher-order construct of calling or to a different model that does not share the same facets.